1,784,016 itemsWelcome to Web Crawls

The Web Archive of the Internet Archive started in late 1996 is made available through the Wayback Machine, and some collections are available in bulk to researchers.

Other than the pages collected by the Internet Archive, major contributors include Alexa Internet, Cuil, and those listed below.

All items (most recently added first) -

Sub-Collections

	Accelovation Crawl Web crawl snapshots generously donated from Accelovation. This data is currently not publicly accessible. From the site: Accelovation is pioneering the delivery of Insight Discovery™ software...	1,324 items
	Alexa Crawls Crawl data donated by Alexa Internet. This data is currently not publicly accessible. Decryption Keys are kept in an item. Alexa is the leading provider of free, global web metrics. Search Alexa to...	92,090 items
	Archive Team Formed in 2009, the Archive Team (not to be confused with the archive.org Archive-It Team) is a rogue archivist collective dedicated to saving copies of rapidly dying or deleted websites for the...	14,102 items
	Archive-It Digital Collection The Archive-It Digital Collection	53,509 items
	Away From Keyboard Away From Keyboard is a memorial collection dedicated to preserving pieces of lives lived online from being scattered and lost. While no collection of data can ever replace a person, these archives...	293 items
	collections-aaron-swartz from Wikipedia: Aaron Hillel Swartz (November 8, 1986 – January 11, 2013) was an American computer programmer, writer, political organizer and Internet activist. Swartz was involved in the...	2 items
	Common Crawl Web crawl data from Common Crawl.	440 items
	Cuil Crawl Data Web crawl snapshot generously donated from cuil.com. This collection of pages mostly from 2007 and some from 2008, is about 310 terabytes of compressed data, and almost 60 billion URLs (mostly text)....	26,494 items
	Custom Crawl Services National library harvesting.	25,208 items
	Focused Crawls Focused crawls are collections of frequently-updated webcrawl data from narrow (as opposed to broad or wide) web crawls, often focused on a single domain or subdomain.	9,958 items
	http://abbotsford.craigslist.ca	4 items
	httparchive Successful societies and institutions recognize the need to record their history - this provides a way to review the past, find explanations for current behavior, and spot emerging trends. In...	27 items
	Institut national de l’audiovisuel Crawl data from Institut national de l’audiovisuel in France. This data is currently not publicly accessible. from Wikipedia: The Institut national de l'audiovisuel (or INA, French for National...	50 items
	Internet Archive Web Crawls Crawl data collected by the Internet Archive. This data is currently not publicly accessible in this format. To view archived web pages, please visit the Wayback Machine.	377,622 items
	Internet Memory Foundation Data crawled on behalf of Internet Memory Foundation. This data is currently not publicly accessible. from Wikipedia: The Internet Memory Foundation (formerly the European Archive Foundation) is a...	50 items
	Mercator Crawl Crawl done with the DEC/HP-labs 'Mercator' crawler and converted to ARC format. This data is currently not publicly accessible.	1 items
	Rescue Crawls Rescue crawls conducted by the public for sites that have announced that they are closing.	2 items
	Thumper Transfer Web crawl data transferred from thumpers in Santa Clara data center.
	urlteam Web Crawls Crawl data collected by the urlteam. The URLTeam is the ArchiveTeam subcommittee on URL shorteners. We believe that they pose a serious threat to the internet's integrity. If one of them dies, gets...	4 items
	Web Collections Web Collections organized by year. Some of this data is currently not publicly accessible.	19 items
	web-group-internal miscellaneous data	28,760 items
	Wiki Collections Collections of Wiki data	48,087 items
	Wikileaks.org Archive A collection of web pages from the wikileaks websites as well as news coverage and commentary surrounding the Wikileaks releases. It includes coverage of the Afghan war diaries, the Iraq war logs,...	3 items